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ABSTRACT 

The monograph describes various aspects of data 
collection, analysis and data-based decision-making for the regular 
or special education classroom teacher. Direct and repeated 
measurement of student progress and program effectiveness is 
advocated. Four fundamental steps are outlined: (1) identify 
instructional objectives and collect data on student performance; (2) 
display the data graphically; (3) examine the data and identify 
trends; and (4) use basic data decision rules to guide instructional 
decisions. The concept of "stages of learning*' is introduced in 
relation to use of percent and rate data. Use of correct-to-error 
ratios and trials-at-criterion standards for setting performance 
goals is presented. Strategies for determining objectively when and 
how to change instruction are discussed. Instructional modification 
involves changing one or more of four basic program elements: general 
setting events, antecedent events, subsequent events, and the 
contingency arrangement. Finally, guidelines for implementing 
data-based instruction and making data-based decisions with students 
who are low achieving, exceptional and at-risk for dropping out are 
offered. (Author/DB) 



X Reproductions supplied by EDRS are the best that can be made X 
X from the original document. X 



ERIC 



CD 
fVl 

uJ 



Implementing Data Based Decisions for Instruction 



Jo M. Hendrickson, University of Florida 
Department of Pediatrics 



U S DEPARTMENT or EDUCATION 

Offtce ot Educational Research and Improvement 

EDUCATIONAL RESOURCES INFORMATION 
CENTER (ERIC) 



his document has been reproduced as 
eceived Uom the person or organ»zation 

on\^inating i| 

r Minor cfianges have been made to improve 
reproduction quality 

• Points of viev^ or opinions stated in thisdocu 
ment do rv)t necessarily represent official 

OERi position or policy 



Robert A. Cable, Old Dominion University 
Department of Special Education 

William Wolking, University of Florida 
Department of Special Education 



^This monograph is reprinted v^ith the permission of the authors and may 
be obtained on a limited basis b/ requesting, "A Guide to Data-based 
Decisions for Teaching Students with Learning Difficulties," from Jo 
M. Hendrickson, J282 Health Center, Gainesville, Florida 32610. 



"PERMISSION TO REPRODUCE THIS 
MATERIAL HAS BEEN GRANTED BY 



fi'^STWY AVAILABLE 




TO THc EDUCATIOrQAL RESOURCES 
INFORMATION CENTER (ERIC)" 



Paper presented at the Annual Convention of the 

Council for Exceptional Children 
(66th, Washington, DC, March 28-April 1, 1988) 



3 



PREFACE 



The merit of conducting direct and repeated measurement on 
academic performance is well documented. Nonetheless, many teachers do 
not apply knowledge gleaned from such measures to make day-to-day 
instructional'decisions. This monograph describes various aspects of data 
collection, analysis and data-based decision-making for the classroom 
teacher Direct and repeated measurement of student progress and 
program effectiveness is advocated. The concept of "stages of learning" is 
introduced in relation to use of percent and rate data. Use of 
correct-to-error ratios and trials-at-critenon standards for setting 
performance goals is presented Strategies for determining objectively 
when and how to change instruction are discussed. Finally, guidelines for 
implementing data-based instruction and making data-based decisions 
with students who are low achieving, exceptional and at-risk for dropout 
are offered. 
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A Guide to Data-Based Decisions for 
Teaching Students with Learning Difficulties 



By most accounts, direct and repeated measures are viewed as 
essential to sensitive evaluation (Fuchs, 1986) and effective instructional 
programming -for students evidencing a range of learning difficulties (e.g., 
Haring & Krug, 1975; Lovitt, 1977; Smith, 1981; White & Haring, 1980). 
Available literature on academic achievement amply demonstrates the 
efficacy of direct measurement procedures. An impressive body of 
evidence documents direct and repeated measurement as "one of the few 
variables that reliably predicts growth as a function of teacher 
effectiveness" (Sailor & Guess, 1983). Use of direct and repeated 
measures is known to positively affect student achievement and is 
recognized as fundamental to effective curriculum-based assessment. By 
comparison, ill defined goals and occasional measurement of student 
performance are major reasons for the failure of classroom instruction 
(Deno&Mirkin, 1977). 

In spite of overwhelming clinical and research evidence, direct and 
repeated measurement of student performance for the purpose of verifying 
the worth of an instructional program is not common practice in education 
(Tawney & Gast, 1984). While many special education teachers (and some 
regular educators) are aware that direct and continuous, curriculum-based 
measurement strategies can be more useful than traditional evaluation 
approaches, few use them. Part of the lack of data-based instruction by 
classroom teachers may be that infcrmauon which will assist 
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practitioners in establishing workable programs and in making data-based 
instructional decisions is scattered and not widely disseminated. Further, 
information on data decision strategies is not well understood and often 
perceived as impractical by the individual teacher. 

This monograph is designed to help bridge-the-gap between research 
and practice -and to enable the classroom teacher to employ data-based 
decisions to assess student progress and to evaluate instruction. 
Following a brief overview of the problem, a synopsis of how to establish 
a data-based instructional program is presented. The concept of "stages 
of learning" is discussed ?s well as teacher selection and use of percent or 
rate data in relation to instructional goals and the student's stage of 
learning. The correct-to-error ratio and trials-at-criterion approaches 
for determining when to make phase changes (e.g., modify instruction) in 
an academic program are presented. Guidelines are provided for how to 
modify instruction once a phase change is indicated. Teachers are 
encouraged to establish local normative standards. Finally, a set of 
guidelines for implementing data-based instruction and data-based 
decision-making is offered to the practitioner. 

Direct and Continuous Measurement: The Problem 

The value of direct and continuous measures of student performance 
for the purpose of individualizing instruction, evaluating student progress, 
and assessing the merits of different instructional approaches is well 
documented (Lovitt, 1977). Nonetheless, routine collection of performance 
data for empirically verifying the worth of instruction and making 
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instructional decisions is not common practice among eJuoators (West, 
1084) Wesson, King and Deno (1984), for example, found that 32% of 
special educators polled expressed knowledge of direct and repeated 
measurement, while only 44% indicated that they actually applied these 
procedures Tindal ( 1 985) reports that even when continuous measurement 
systems are employed, teachers do not institute adjustments in 
instruction with sufficient frequency. Moreover, direct, frequent 
measures of student performance do not in themselves ensure sound 
instructional decision-making (Tindal, 1985). Simply stated, direct and 
repeated measurement is not functional unless instructional outcomes 
(i.e., data on student performance) control the actual implementation of 
instructional decisions (Eaton, 1978). In short, although quality 
day-to-day instruction is predicated on frequent evaluation of student 
performance (Englert, 1984), widespread implementation of the 
fundamental state-of-the-art assessment procedures remains unachieved. 

The Data-Based Classroom 

Data-based instruction can be carried out successfully in- (a) 
elementar/, middle or secondary level classrooms, (b) regular education, 
special education and alternative education programs, and (c) large group, 
small group and independent teaching arrangements. To establish 
data-based classroom practices, four fundamental steps will need to be 
carried out by the teacher: (1) identify instructional objectives and 
collect data on student performance, (2) display the data graphically, (3) 
examine the data and identify trends, and (4) use basic data decision rules 
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to guide instructional decisions. These steps are described briefly below 
and are followed by suggestions for making data-based classroom 
decisions. 

Step 1. Identify objectives and collect data. In order to evaluate 
whether a specific objective is being achieved within an expected time 
frame (or has been mastered), it is necessary for the teacher to delineate 
precisely the instructional task of interest. Fortunately, the majority of 
academic behaviors taught in our schools are amenable to clear definition 
and objective evaluation. Direct and repeated measurement of simple 
academic behaviors such as oral reading, writing words in response to 
story starters, and writing spelling words is known to yield valid, reliable 
indices of student performance (Fuchs, 1986). Accordingly, teachers may 
target dependent variables such as these w'th confidence. 

Once instructional objectives have been targeted, defined and 
prioritized, data on student performance can be gathered. Jenkins, Deno 
and Mirkin (1979) propose that an effective data collection system must, 
(a) yield a fair representation of overall pupil performance, (b) be 
sensitive to small changes in performance, (c) be flexible and adaptable to 
various instructional objectives, and (d) be repeatedly and easily 
administrable. Evans, Evans and Mercer (1985; recommend the use of 
probes (ie, brief structured, timed tests) administered subsequent to 
instruction. Timed probes meet the above-mentioned criteria for an 
effective data collection system. Fuchs (1986) notes'that 1 to 3 minute 
timings (i.e., probes) are practical to administer and technically sound. By 
systematically controlling the length of the probe, the schedule of the 
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probe session, the type of student response, and format and content of the 
probe itself, teachers can make reliable comparisons of student 
performance across time and among students Assessment of student 
performance through the use of brief probes can be accomplished 
strategically in different ways—probes nay be teacher administered to 
individuals or groups, students can administer probes to one another, and 
students can learn to self-administer probes (see sample probe sheets in 
Figure 1) 

Step 2: Display the data graphically. Each time data are collected on 
a student's performance using a timed probe, the data should be plotted on 
a graph Charting 'lata on a graph proyides immediate visual feedback on 
studenc performance and enables visual inspection and analysis of the 
effects of instruction. Two main charting conventions, equal interval and 
logarithmic paper, are used in education. Examples of equal interval and 
logarithmic graph paper are presented in fiQures 2a and 2b, respectively, 
with the Siame hypothetical sets of data charted on each. 

While the literature clearly substantiates the benefits of data 
graphing, it fails to support unequivocally one charting convention over the 
other (Fuchs, 1986). Tawney and Gast (1984) argue that equal interval 
paper facilitates data analysis and is easier for practitioners to 
understand (e.g. Figure 2a). Limited investigation of the predictive 
capacities of the two graphing procedures suggest that a trend line (i e., a 
"best fit" line that represents the slope of the data) on equal interval 
paper may be equally or more accurate than those recorded on logarithmic 
paper (Fuchs, 1986). However, as West (1984) points out, a logarithmic 
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Figure 1 
Sample Probe Sheets 
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Figure 2a 

FouGl Interval Graph Paper : Number of Correct and Incorrect Words 
ReaC Orally In One Miaute Under Two Intervention Conditions 




Figure 2b 

Logartlhmic Graph Paper : Number of Correct and Incorrect Words 
Read Orally in One Minute Under Two Intervention Conditions 



scale (also referred to as a proportional scale) enables tne display of a 
wider range of scores (see Figure 2b), does not distort relative 
equivalence in scores (compare pictures of Intervention I and 2 in figures 
2a and 2b), yields a straight or nearly straight line which can serve as the 
growth/trend line, and provides a picture of relative changes in learning 
which can be compared directly to one another. 

Given present research on charting data, selection of a particular type 
of graph paper may best be resolved as a matter of teacher preference 
(Gable, Hendrickson, Evans & Evans, in press). Most of the natural 
sciences, for example, have decided in favor of logrithmic paper while the 
education community traditionally has employed equal interval graph 
paper Frequent use of either chart, particularly in settings serving a 
large number of students can be made more efficient by: (a) teaching the 
students to graph their own performance (and the performance of peers) 
and (b) setting up a data management station for scoring probes, charting 
performance, and interpreting instructional outcomes 

Step 5 Examine the data and identify trends . Visual inspection of 
frequency data usually reveals one of three basic performance trends 
These three trends are used to describe correct responses and incorrect 
responses Once a sufficient sample of student performance has been 
gathered the three trends typically seen are. ( 1 ) an increasing/ celerating 
trend, (2) a decreasing/decelerating trend, or (3; a stable state Figure 4 
depicts common trend lines and combinations of correct and error slopes. 
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Figure 3 

Common Trend Lines and Descriptions 



Trend 



Descriati on 



increasing/cclerating 



decreesing/decelerating 



stable/steady 



variable/not under 
instructional control 





increasing corrects 
decreasing errors 



increasing corrects 
increasing errors 



cfecreasing corrects 
Increasing errors 
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Once trend lines are identified, the teacher has a solid basis for 
making data-based decisions In addition to examining the trends in the 
data, teachers may use data variabili[;', steepness of the trend/slope 
lines, and student performance on the most recent instructional days to 
determine the effectiveness of instruction [Readers are referred to 
Haring, Lovitt, Eston and Hansen (1978), McGreevy (1986), West (1984), 
White (1985) and White and Liberty (1976) for greater detail and 
discission of charting and analyzing dir3ct and continuous measurement 
data.] 

Step 4 Use basic data decisions rules Kerr and Nelson ( 1983) oefine 
data-based decision making as "using direct and frequent measures of a 
behavior as a basis for comparing student performance to a desired level, 
and making adjustments in the students educational program based cn that 
comparison" (p. 344). Decision rules allow practitioners to examine data 
on student performance and determine when to make program revisions 
(Eaton, 1978). To help the teacher make sound instructional decisions, 
basic rules for interpreting trends in the data are useful; Figure 4 provides 
basic decision rules in relation to common data trends. 
Stages of Learning 

McGreevy ('.986) states that learning can be defined as a change in 
performance across time Lovitt and his colleagues have proposed con- 
ceptualizing learning as a process which includes stages ranging from 
acquisition to adaption (Haring et al :978). By employing this 
conceptualization in the classroom, teachers may be better able to plan 
for and assess the outcomes of instruction 
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Figure 4 

Common Trends and Decision Rules 



Trend 



Decisibii 




The divergence of correct and error 
trends is the desired outcome of instruction » 
and indicates good discrimination of stimuli 
and differential reinforcement of desired 
behavior. 






Horizontal and parallel trends mean that 
a discrimination problem possibly exists. 

Parallel rising trend lines indicate the 
possibility that errors are being reinforced. 

Parallel falling trends indicate the 
possibility of extinction conditions. 

if correct and error trends diverge 
to a large degree and then run parallel, 
there is the possibility of a schedule of 
reinforcement problem. 

Convergence of error and correct trends 
indicates the possibility of discrimination 
problems ana of noncontingent reinforcement. 

A rising correct trend with an even more 
sharply rising error trend possibly 
indicates a contingency problem arising 
from subjects hurrying to get finished. 



Source- Burney(1976) 
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West (1984) defines learning as a change in performance and also 
emphasizes that learning is a process. Correspondingly, he raises the 
thought-provoking question of what dimension of performance must change 
in order for educators to agree that learning has occurred? While 
educators predominantly use percent correct (i.e., accuracy) as the primary 
proof of learning, accuracy may be viewed as a qualitative (not quantita- 
tive) measure of performance (West, 1984). To obtain quantitative 
measures of performance West recommends the use of rate data. 

Based on the research and clinical observations of Lovitt, McGreevy, 
West and others, a Stages of Learning: Measurement Decision Chart (see 
Table 1 ) was designed. This Measurement Decision Chart interrelates four 
stages of learning (column 1) with the goal of instruction at each stage 
(column 2) and provides the practitioner Vv^ith a recommended measure for 
assessing student performance (column 3). Column 4 states the source 
from which the standard was obtained. 

As Table 1 illustrates, academic progress can be conceptualized as a 
stepwise advance through four learning stages ranging from response 
acquisition to proficiency, maintenance and generalization. While 
empirical data do not validate a rigid learning sequence, it is helpful to 
classify student learning according to stages in order to select the unit of 
measurement, determine when to modify instruction, and decide how to 
adjust intervention techniques. 

At the beginning stage of learning, the main objective is for the 
student to demonstrate correct performance of new skills or knowledge 
(eg, to add correctly facts w'th sums to 10). Next, during proficiency— 
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stage 2, the goal of instruction shifts to include speed as an important 
dimension of performance (eg, the teacher now requires the student to 
add correctly 50 facts in one minute). In stages 3 and A, maintenance 

^3^'® Stages of Learning: Measurement Decision rhart 



Learning 



2- Proficiency 
(Fluency Building) 



3- Maintenance 
(Durability) 



Goal 



1 - Acquisition Accuracy 
(initial Learning) 



Accuracy 
& Fluency 



Retention 
Endurance 



4- Generalization Expansion 
(Adaptability) Extension 



Measure Standard 



PERCENT 
Ratios cor- 
rects errors 
Trials-at- 
Criterion 

RATE 

Trials-at- 
Criterion 



RATE 



PERCENT or 
RATE 



Expert judgment 
Functional analysis 
of "critical effect" 



Peer referents 
Adult: child formulae 
Compare oast-to- 
present mastery 

Peer referents 
Adult: child formulae 
Compare past-to- 
present mastery 

Mainstream norms 
Community norms 



Adapted from Hanng, Lovitt, Eaton & Hansen ( 1978); Smith (198 I) 

(response durability) and generalization (response adaptability) are the 
aim of instruc::ion. During maintenance the teacher is interested in 
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whether the level of fluency attained during proficiency building will be 
retained 6, 12 or more weeks later. During generalization the teacher may 
be interested in response application in new circumstances (e.g., compute- 
generated addition facts w;th sums to 10).' Additionally, a student's 
adaptation and/or modification of a response and its transfer to a novel 
context (e.g.-, solving a math puzzle or word problem through the 
application of addition with sums to 10) is a goal of generalization. 

In sum, use of percent data and correct-to-error ratios are advocated 
during acquisition (refer to Table 1). A trials-at-criterion standard may 
be useful during acquisition and proficiency. Rate ''ata are preferable for 
assessing proficiency and maintenance of pupil responses. For measuring 
performance at the generalization stage of learning, the measure applied 
will vary, depending on the exact nature of the task. Further discussion of 
issues related to percent and rate data follows; the utility of 
corrects-to-errors ratios and tnals-at-criterion standards with direct 
and repeated measurement systems are presented. 

Percent Data and Decision-Making 

Percent data are used extensively in education even though rate 
measures may be the most functional indices of pupil performance (Gable, 
McConnell & Nelson, 1985). Nonetheless, use of percentage is advocated in 
certain instances. For example, at the acquisition stage of learning the 
objective is for the student to perform the task correctly. In striving for 
accuracy the teacher concentrates on response correctness and not on the 
rapidity with which it is executed. Indeed, teacners often direct learners 
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10 take their time and get it right during acquisition (Hendrickson, Gable 
& Stowitschek, 1985) 

In some instructional arrangements, students may be restricted from 
responding as rapidly as they actually are able (e.g , when spelling words 
are dictated by the teacher) (Eaton, 1978). When there is a ceiling 
imposed on response rate, percent data may be the best measure of student 
learning. Similarly, complex tasks/problems requiring generalization may 
be more amenable to percent than rate measures (Hendrickson et al , 
1985). For example, the teacher may assign students 1 to 3 complex 
problems to solve and allocate as much time as needed to solve them, in 
such a case accuracy in application of conceptual knowledge and the 
execution of different subtasks may be more important instructionally 
than solving numerous problems in a short time. 

Establishing percent criterion standards Haring and his colleagues 
(1978) report a procedure for establishing a percent growth criterion 
against which to judge the impact of instruction (Haring, Lovitt, Eaton, 
Hansen. 1978) This procedure is recommended particularly for instruc- 
tion that involves the acquisition of new skills. To 'mplement the 
procedure, three initial measures are obtained and data on the number of 
items correct are charted (e.g., 6, 10, and 8 correct addition problems) 
Once the three data points representing student performance are plottec, 
the median or mid-frequency score, is used as the basis for setting the 
instructional aim for the next 3 days. For example, if- 8 represents the 
median score, the teacher can establish an instructional goal by selecting 
a minimum percentage representing an acceptable increase in student 
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performance (eg., 30 %) Next, the median is multiplied by the growth 
factor selected by the teacher (i.e., 8 x 3 = 2 4). Finally, the product is 
added to the origir.ai median (8 + 2.4 = 10.4) to yield a criterion standard 
for number of correct responses expected on each of the subsequent 3 
teaching sessions. Pupil performance at or above 10.4 indicates that 
student progress is acceptable within the current instructional program. 
In contrast, should 1 - 3 data points fall below the newly specified 
criterion (e.g, 8, 9, and 10), the instructional program would be 
reassessed and probably modified. 

When an instructional aim is almost achieved (e.g., 10, 9, andl I), an 
additional I - 3 data days (i.e., teaching sessions) should be conducted to 
substantiate this modest subaverage trend before a phase change is made 
in the instructional program If a ooor growth trend is substantiated the 
instructional program then would De modified. This procedure for setting 
percent performance standards is repeated about every three sessions 
until the terminal ins.tructional goal is attained It should be pointed out 
that probe data need not be collected each day of instruction; however, 
three or more times a week is recommended (McGreevey, 1986, West, 1S84). 

Other procedures for establishing percent criteria. A minimum 
standard of 83 percent accuracy on academic tasks is advocated by some 
authorities (Howell & McCollum-Gahley, 1986). Stephens ( 1977) suggests 
using a variable percent criterion to determine the entry level of the 
student (as well as setting learning standards). His learning levels 
include- (a) mastery a" 100% correct, (b) learned at 90-99% correct, (c) 
instructional at 70-90% correct, and (d) frustration at 69% or Ie5:3 
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correct By demanding that the student achieve a predetermined criterc^r, 
such as 83% correct across teaching trials, the likelihood increases that 
instruction will be successful (i e., be retaified and generalize) 

Table 2 contains a guide to making phase change decisions. Making 
phase chanyes (i.e., program modifications) is presented as a basic 4 step 
process Step" 1 is establishing the performance criterion standard against 

Table 2: Four Step Guide to Phase Change Decisions 



Step 



Activity 



Establish performance criterion standard 



(a) percent standard 

(b) rate standard 

vc) ratio of ccrrects-to-errors standard 

(d) trials-at-criterion standard 



2 



Evaluate student performance across three (to 
ten) teaching sessions 



3 



Judge instruction based on the criterion 
standard and/or a revised criterion standard 



4 



Maintain, modify or terminate instruction 
(Based on i aton, 1978; 
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which the student's growth can be measured. Establishing a criterion 
level IS done regardless of the measure employed (e.g., percent or rate) 
(Examples for setting up percent criteria were described earlier in this 
monograph.) Step 2 consists of evaluating student performance to 
ascertain whether the criterion is met. Step 3 requires judging instruc- 
tion based on the criterion standa.^d established. Step 4 consists of 
implementing the decision to maintain, modify or terminate the 
instructional program. As stated, this sequence of steps for making phase 
change decisions applies regardless of the type of measure employed to 
assess student performance 

Rate Data and Decision-Making 

As previously discussed, by using percent alone, knowledge is gained 
on the degree of accuracy with which a task or steps of a task is/are 
performed Rate data— frequency of responses in relation to time, go a 
step further by including response frequency and speed in the measurement 
process Two other temporal measures of behavior, latency and duration, 
are needed less often by teachers and consequently are not discussed. 
Teachers should note, however, that latency and frequency have an inverse 
relationship in that the greater the amount of time neeaed to respond, the 
fewer responses a student is able to make. The steps shown in Table 2 
will be employed when percent, rate, correct-to-error ratios and tnals- 
at-criterion standa^^ds are used. 

E stablishing ra te criterion standards. Once plotted (typically on 
logarithmic paoer), rate data enable teachers to readily assess the slope 
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and Vcjnability of pupil performance (Deno, Mor?ton & Tindal, 1986). 
Academic/instructional placement decisions based on rate typically are 
predicated on the teacher obtaining a median performance score and 
drawing a slope or trend line on the student's chart The median and slope 
should not be calculated until 6 to 10 days of probe data have been 
collected (Deno et al , 1986; Lindsley, 1971; Sailor & Guess, 1983). 
[Evaluation of percent data, it may be recalled, can be conducted with as 
few as 3 data days although some authors also use 3 data days w.th rate 
data (eg., Haring et al., 1978).] Once initial measures (baseline data) are 
collected and analyzed, direct and frequent measurement (and graphing) 
should be continued to aid the practitioner in judging the instructional 
effectiveness of a particular prog, d;.-, (Deno et al , 1986; Evans & Evans, 
1 985; West, 1 984) 

A desired growth rate can be established by the teacher based on 6 to 
10 non-zero data points (Deno et al., 1986; Sailor & Guess, 1983) Detailed 
procedural descriptions are available elsewhere regarding how to draw 
trend/growth lines. The short explanation for deriving a growth line that 
follows IS offered more to "demystify" the process than to fully explain 
the procedure. 

First, to establish a desired growth line~a criterion standard against 
which teachers can judge the instructional effectiveness of a program, at 
least 6 data points must be plotted. The data points on the graph are 
halved Next, the mid- point of the data in the first tialf is obtained by 
finding the mid- rank score (i.e., by ordering the scores from lowest to 
highest) and then locating the mid-day. data point a plus sign (+) is 
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recorded on the graph at this so-called mid- point. the point where the 
mid-day and mid-rank point intersect. This procedure is repeated for the 
second half of the data. The two plus signs (+) recorded at both 
mid-points are connected to yield the slope (or trend line). This trend line 
can be extended and serve as a growth criterion for future performance. 
Teachers can evaluate the future impact of instruction and subsequent 
modifications in in-truction by simply inspecting and comparing 
corresponding student performance data (i.e., differing slopes) to the 
growth line (Deno et al, 1986; Haring et al , 1978; Haring, Liberty & White, 
1980) 

Decisions on when to modify instruction Ci e., phase change) also may 
be based on the amount of variability in student performance. A "window 
of tolerance" (or variability) can be drawn around the growth line to give a 
a visual boundary to how much variability in performance is acceptable. 
To create this window, one line is drawn parallel to and above the growth 
line A second line equidistance, parallel to and below the growlh line is 
made next, if 10 data points, for example, were used to estimate a 
growth line, at least 3 - 4 points should fall between the top line and the 
growth line Another 3 - 4 points should fall between the growth line and 
the bottom line (see Figure 5). Generally speaking, the narrower the 
window and the more data points within the window, the greater the 
instructional control. In Figure 5, all data points fall within the window 
of tolerance- a phase change would rot be necessary. ' 

On the other hand, if the window is very wide or many data points 
(e.g., 4 of 10) fall outside the window, program modifications may be 
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needed A legitimate instructional goal simply may be to reduce vari- 
ability in student performance Teachers .may construct transparent 
overlays that contain a series of parallel lines which signify a set of 
different sized "windows of tolerance" The smaller windows, of course, 
representing more stable data and greater instructional control. Finally, 
in the absence of peer standards teachers can experiment by making 
program changes after collecting and plotting the data. The relative 
effectiveness of each succeeding phase change is judged according to the 
angle (steepness) of the slope. 



Figure 5 
Sample Window of Tolerance 
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Correct-To-Error Ratio and Decision-Making 

During initial acquisition of new skills/knowledge a ratio of the 
number corrects to number of errors can be very useful to the teacher 
Correct-to-error ratios such as 10.21, 20: 18; an 5.16 clearly represent 
performance with an easily recognized unaccept..- ' high error percentage 
(67 7%, 47.4%, and 76.2% of the total responses, respectively). By 
examining the number of corrects in relation to the number of errors and 
the errors in relation to the total number of responses completed, the 
teacher can gain a good clinical grasp of the student's learning stage. The 
higher the number of errors in relation to corrects, the more likely the 
student is experiencing frustration—the instructions! program may need 
to be sliced back to an easier level When correct to error ratios decrease 
to less than 10% of the total responses (e.g., 10:1, 24:2, and 132:2), it 
signifies that the student is advancing from acquisition toward pro- 
ficiency As such points are identified, the teacher will focus increas- 
ingly on speed of performance. 

Establishing correct-to-error ratio standards Suffice it to say, by 
scrutinizing the ratio of corrects-to-errors the teacher can make a quick 
estimation of whether the material corresponds to a frustration, 
instructional or independent level of performance for the student. The 
actual correct-to-error ratio standard that a teacher may establish before 
making a program more demanding, for example, would include 
consideration of; (a) the topography of behavior, (b) the importance of that 
behavior to specific skill/knowledge acquisition i;i the future, (c) the 
Importance of that behavior to gene'-^l school success, and (d) the amount 
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of time likely to be required to teach to a stricter criterion level. By 
examining correct-to-error ratios and modifying instruction accordingly, 
the teacher can avoid frustrating the student and unwittingly promoting 
avoidance-motivated behavior 
Trials-At-Criterio n and np cision-Making 

The main purpose of measuring academic performance is to assess 
achievement and 'earning and thereafter to design or modify instructional 
programs in accordance v^ith the educational needs of the student(s). 
Traditionally, an arbitrarily imposed percent correct figure (e.g., a single 
demonstration of 80% correct) has been us^d as a standard for learning 
(White ex Liberty, 1976) Huv^ever, as White and Liberty (1976) assert, the 
criterion level selected should define the minimum acceptable 
performance v^hich facilitates learning subsequent skills In that fev^ lov^ 
performing or exceptional students are "one-trial" learners-able to 
master a skill on the first presentation, the introduction of a 
trials-at-criterion standard may be more functional than a single percent 
correct score. That is, "x" number of correct responses across "y" number 
of teaching sessions may constitute a more suitable criterion statement 
of mastery than a single percent correct score. Unfortunately, guidelines 
v/hereby teachers are able to establish optimal criterion standards for any 
given skill or content area do not exist. Indeed, it is likely that functional 
standards may vary from student to student, with conservative (i.e, high) 
standards of mastery recommended for students experiencing difficulty. 

Establishin g trials-at-criterion standards Traditionally, the 
literature refers to trials-to-criterion as a basis for determining if a 
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skill IS mastered We are employing the term trials-at-cntenon to 
underscore the importance of a repeated performance standard across 
days Since the majority of underachievers, exceptional learners and 
students at-risk for dropout are academically delayed, a trials-at- 
cntenon standard is advocated because it incorporates the concept of 
overlearning ..(i.e., a high percent/rate correct repeated across trials 
and/ or days, ivarie, 1986). When employing trials-at-criterion with 
percent measures, caution must be exercised so that a fixed common 
demoninator (i.e., equal number of opportunities to respond) is maintained 
across sessions Too few opportunities to respond probably will not hold 
educational significance, regardless of the standard imposed. In thp final 
analysis, the worth of any criterion standard must be judged according to 
its "critical effect"— the impact on present as well as future pupil 
performance. Teachers should try to identify standards which are most 
efficient (i e., the least time-consuming) and produce the greatest desired 
effect. For this reason, the decision of how stringent to make a given 
trials-at-criterion standard is best made on a pupil-specific basis 

As mentioned, one aspect of evaluating "critical effect" is to 
establish criteria on certain academic (sub)skills and evaluate their 
affect on students" learning of higher level skills (White & Liberty, 1976) 
For example, a teacher could assess the affect of a trials-at-criterion 
standard of 3 and 6 trials on individual and/or group mastery of addition 
facts After the identified students reach their respective criterion 
level, multiplication facts are introduced. A comparison to determine 
which students (i.e., 3 or 6 trial) master the multiplication facts faster 
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and with greater accuracy would yield information on which standard is 
more useful. Finally, data obtained on student retention of skills/ 
knowledge can be used to guide selection of a trials-ai-criterion standard 
For the instruction of new skills teachers initially can base the selection 
of a criterion level on experience/data from similiar 3reas (e.g., criteria 
for oral sight vocabulary words may be used to assess acquisition of 
spelling skills) 

How to Modify Instructi on 

Instructional programs may be modified in a variety of ways. Without 
doubt, some modifications will be more effective and/or mo^e efficient 
than others. With a data base to guide program modifications the teacher 
should have more confidence in his/her professional judgment. None- 
theless, a data base and decision rules together are not sufficient to 
ensure precise, accurate program modifications. To determine the best 
instructional adaptation, teachers must have some basic knowledge of 
instructional design. While there are many components of good 
instructional design, generally speaking, teachers can manipulate four 
basic elements of any program: (1) the general setting events, (2) the 
antecedent events, (3) the subsequent events, and (4) the contingency 
arrangement. Each of these program components is described below 
followed by more specific guidelines on how to modify instruction when a 
student is performing at a particular stage of learning.' 

General setting events General setting events are those conditions 
under which instruction takes place. Typically, setting Pv^ents do not exert 
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as great an influence on student performance as do antecedent and 
subsequent events or as contingency arrangements Setting events include 
the time of day, noise and lighting conditions of the instructional space, 
the number of people involved, the physical design of the classroom, the 
seating arrangement for a lesson, the type of equipment and materials 
(e.g., print versus an audio-visual presentation) and so on. 

Antecedent events . Antecedent events are those instructional cues, 
prompts, directions and other specific stimuli which come before and 
directly control (or should control) the learner's behavior For example, 
the teacher direction, "Jonathan, read this passage carefully is a specific 
prompt for behavior from Jonathan. Modification of an antecedent event 
might include changing the direction (e.g., "Jonathan, I want you to read 
rapidly and without errors) or altering the actual passage (e.g., dividing 
the passage into three sentence segments or highlighting difficult words). 

Subsequent events. Subsequent events are those events which follow 
a student's performance (very) closely in time. When these events are 
positive they are commonly referred to as reinforcement and/or feedback. 
Subsequent events arranged by the teacher or that exist as part of the 
instructional material, may or may not be equally reinforcing to all 
students Self-correcting materials often make provision for clear and 
prompt subsequent event presentation. Instructional games can be 
designed to give immediate feedback on the correctness of each response. 
Similiarly, most educational software is written to" supply visual and 
Often auditory feedback on each response. In such cases, the subsequent 
event reinforces the correct answer. Another example of a subsequent 
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event is teacher praise or distribution of points provided for correct 
responses as they occur. Conversely, a frown from a teacher or peer could 
be an unplanned (and negative) subsequent event As with general setting 
and antecedent events, subsequent events may be planned or unplanned, 
either way, they are present in any instructional program. The teacher 
who consciously adjusts these events and monitors their affect on 
learning is the teacher who will achieve greater instructional outcomes 
academically, behaviorally and attitudinally. 

Contingency arrangements Contingency arrangements refer to the 
relationship between student performance/responses and the consequent 
events that follow the behavior. With good instruction, the correspond- 
ence between correct responding ana positive reinforcement, for example, 
is systematic and consistent. The exact relationship between behavior and 
its consequence can be varied in numerous ways In general, a "one-to-one 
relationship between behavior and its consequence" is called a continuous 
schedule Consequent events c^n be given at fixed times (e.g., every 5 
minutes) or for fixed amounts of work (eg., 1 star for every 10 problems 
correct) Consequent events also can be delivered on variable schedules. 
In a variable schedule, after an average of x amount of time has elapsed 
(e.g., tne average time is 5 minutes-the actual delivery of reinforcement 
varies from 3 to 7 to 5 to 4 minutes, etc.) or / number cf responses have 
been made (eg, an average of 10 responses earns one a star— stars 
actually are given after 6, 10, 13, 8 responses, etc.), the subsequent event 
IS delivered following the behavior Continuous schedules of positive 
reinforcement are known to be an extremely effective way to promote 
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initial learninv,, whereas variable schedules of reinforcement are 
essentia! for maintaining skills/knowledge across time and are more 
useful for increasing the speed of responding. During acquisition, it may 
be critical for a student to receive a positive consequence (e.g., praise 
statement) after each correct response. The teacher should also have a 
plan for how to consequate each error After the student begins to 
perform the skill with increased correctness, the teacher delivers a 
positive consquence once after every three correct responses. These 
contingency arrangements, 1:1 and 31, represent one teacher consequence 
for each response and 1 teacher consequence for three responses, 
respectively Note that contingency arrangements should be planned for 
both correct and incorrect responses. 

In short, decisions on how to modify instruction may be tied to: (a) 
stages of learning and (b) temporality of events (Gable et al , 1985). Table 
3 contains a guide to making instructional changes in relation to the 
"ti'denfs stage of learning and the instructional events which occur before 
and after a student responds Generally speaking, teachers should be 
cautioned that instructional modifications should be made one at a time. 
The actual impact of programmatic change will be confounded if several 
aspects of instruction are simultaneously manipulated. 

As seen in Table 3, the stages of learning-Stage 1 (acquisition). 
Stage 2 (proficiency) and Stage 3 (maintenance), and Stage 4 
(generalization), are presented along the left hand 'side of the table. 
Antecedent and subsequent events which the teacher may alter are 
presented in two separate columns. If a student or group of students is 



ERIC 



28 

34 



experiencing failure during Stage 1, for exampie, the difficulty of the 
materials and/or provision for models, prompts and advance organizers 
might be a first step to modifying antecedent events. Enriching the 
schedule of reinforcement (e.g, moving from- 10:1 to 4:1 behaviors per 
reinforcer) and/or power of the reinforcement (e.g., switching from giving 
i star to givir 5 minutes of free computer time) are recommended 
subsequent event modifications. Additionally, the teacher modifying 
subsequent events is wise to evaluate the immediacy with which positive 
feedback is provided. A delay in feedback during acquisition is often at 
the root of poor learning. 

During Stages 2 and 3, proficiency building and maintenance, 
antecedent event changes may include adjusting the pace of presentation, 
clarifying the relevance of the content and/or simply instructing 
students to "go faster". Subsequent event modification during these 
stages typically includes weaning ^he schedule of reinforcement, 
increas- ing the amount of work or speed requirea to rec3ive 
reinforcement, intro- ducing corrective feedback on errors and instructing 
students in "self- control/management " 

During Stage 3, generalization, antecedent events which are 
suggested for modification include the introduction of regular class- 
room and/or grade level materials, varying the topography of the expected 
response, varying the type/complexity/medium of the presentation, 
removing direct instruction, and moving into a less restrictive 
environment. Subsequent event manipulation may include such events as 
further weaning of the reinforcement schedule, providing for "naturally 
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Table 3 



Guide to instructio nal Program Changes By Stages of Learning 



Antecedent Events 



Subsequent Events 



^Provide advance organizers 
^Provide specific directions 
^Provide models/prompts 
^Employ self-correcting 
materials 

^Provide controlled proctif^ 

^Change format of instruc- 
tion (e.g., small group to 
one-to-one) 

^Decrease difficulty of 
msterial/lengtn of lesson 

*Use task analysis to 
splice skills 



^Provide consistent 
consequation for errors 

^Reinforce contingently 



*Enrich schedule of 
reinforcement 

* Enhance power of 
reinforcers 

* Increase immediacy 
of positive feedback 



*Model fast responding 


♦Reinforce rate 


^Present pacing signal 




*Modify teaching arrangement 


♦Lean schedule of 


(e.g., small group to computer 


reinforcement 


assisted) 




*Ad)ust pace of presentation 


♦Increase performance 


^Clarify relevance of content 


criteria 


♦Instruct to salient apsects 


♦Introduce corrective 


of desired performance 


feedback on errors 


(e.g., "Go faster.") 




*Teach self-monitoring, 




*Remove models/prompts 


♦Introduce self-recording, 


*Provide independent practice 


self-evaluation, 


♦Increase opportunity to 


self- reinforcement 


practice 


♦Teaqh goal setting 
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"'"able 3: Guide to In structiongi Program Changes By Stages of 
Learning (Continu ed) 



# \i i ^ 1^ W w Vi w 1 1 w L.T^IlLwl 


DuDsequent hvents 


♦Alternate format of instruction 


♦Further lean the 


(e.g., games, tutorial) 


schedule of 


♦Introduce regular classroom 


reinforcement 


instructional material 


♦Review student perform- 


*lntroduc8 large group 


ance periodically/ provide 


instruction 


specific feedback 


*Vary topography of response 


♦Provide for naturally 


required 


occurring reinforcement 


♦Vary type/complexity/ 


♦Provide sociol 


medium of presentation 


reinforcement 


♦Remove direct Instruction 


♦Provide variable 


*MoYe to less restrictive 


schedule of reinforcement 


environment 




♦Increase problem-solving 




focus of instruction/environment 





occurring" reinforcers, and review and provision of periodic feedback to 
students. 



Stowitschek, Gable and Hendrickson (1980) point out that it is pro- 
bably wise to first modify the subsequent event because materials adapta- 
tion usually requires a greater time investment and expertise on the 
part of teachers than modifying consequences. On the other hand, 
ideally the decision to modify a program should be based on what is most 
likely to improve student outcomes, not its affect on teacher workload. 
Many commerical and teacher-made programs, however, lack provision for 
clear, concise consequation of pupil performance. Therefore, modifying 
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subsequent events remains a good initial step to program modification, 
particularly during the proficiency and maintenance stages. Even so, given 
the limited empirical evidence to support specific program changes 
(Fuchs, 1986), the options shown in Table 3 "are presented as tentative 
recommeMdations. Teachers are advised to establish and maintain files 
with copies of program modifications that were especially effective By 
separating graphs and program descriptions by content or skill areas, the 
teacher can accumulate a personal library of proven instructional 
modifications from which to draw on for future instructional decisions. 

Use of Normative Standards 

Vet another facet to instructional decision -making is the use of 
normative peer-referenced academic standards. By noting on each graph a 
peer performance standard, it becomes possible to judge the student with 
mild to moderate performance deficits against the performance of peers 
who are succeeding on tasks which the student aspires to master (e.g., 
mainstream classroom peer performance in grade level materials). Peer 
performance standards may be established in relation to one 
representative peer or a group of peers. Caution must be exercised in 
using the performance data of a single peer to set criterion standards for 
the target student. Each child is unique and his/her learning pattern may 
not be comparable to the target student on a given skill or at a given point 
in time; on the other hand, the average performance of peers may be a more 
reasonable aim or standard for underachieving students. Teachers may 
gather data on peers in their own classrooms or from other classrooms in 
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the building Peer performance data collected several times a year will 
help the teacher know if his students are within the range of acceptability 
for that school or school district. Knowledge and utilization of local 
normative standards, however, must be tempered by the realization that 
"teacher tolerance" and expectations vary across teachers and from 
student to student (Evans et al., 1986). 

Guidelines for Establishi ng a Data-Based Classroom and Making Data-Based 
instructional Decisions 

The majority of preservice teachers are taught to examine the impact 
of their instruction according to measurable changes in pupil performance 
and achievement (e.g., Kerr & Nelson, 1983). From a practitioner's 
viewpoint, one obstacle to using data-based procedures is the belief that 
such procedures are too time-consuming (Wesson et al., 1984) Further, 
data-based programs are believed to take time away from teaching to test 
(Sailor & Guess, 1983). Additionally, the fact that few teachers engage in 
the use of direct and repeated measures and data charting may be 
attributable to an absence of reinforcement for becoming effective data 
managers Finally, the lack of guidelines to assist teachers in carrying out 
the complex task of addressing the educational needs of a heterogenous 
group of students is apparent. Therefore, the following recommendations 
for establishing a data-based program and implementing data-based 
decisions are reiterated to facilitate the efforts of teachers who desire to 
bring the virtues of quality data analysis into practice in their 
classrooms: 
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The Data-Based Classroom Program 

1 Begin with one student (and one subject area) Set 
performance standards Collect data, chart and analyze 
performance Make data-based decisions 

2 Collect data only on essential and/or priority skills 

3. Graphically depict data immediately (or as soon as possible) 
after measuring performance and collecting the data 

4 Learning may be conceptualized as progressing through the 
stages of: acquisition, proficiency building, maintenance 
and generalization The learning stage at which a student is 
performing should be used to select the performance measure 

5 Establish a data management station for scoring, charting 
and providing students feedback on their performance. 

6. Administer written probes to groups of students. Teach students 
to administer probes to oth-^r students. Teach students to 
administer their own probes. 

7. Teach students to chart their own performance and set their 
own performance objectives. Volunteers and parents also can 
be taught to administer probes and collect and chart data. 

8 Share charted data with students, parents and school personnel 
9. Use student charts to support the need for new instructional 

materials, equipment or other student services (e.g., speech). 
1 0 Teach yourself or obtain training on how to become more 

efficient in data collection, charting, analysis and program 

modification. 
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Data-Based Instructional Decisions 

1 Direct and repeated measurement is ttie best assessment 
approach for monitoring student progress and evaluating the 
merit of classroom instruction 

2 Percent and rate data are the student performance measures 
most frequently used in education. Percent data are especially 
useful during the acquisition stage of learning, while rate data 
are essential during proficiency building. 

3 Several data points (usi ally 3 to 1 0) are needed to determine 
whether a phase change is necessary (i.e., whether instruction 
should be modified). 

4 Equal interval or logarithmic graphs may be u"^ed to present data 

5 Growth and/or trend lines on student charts can be used jy 
teachers to establish performance standards 

6. Both correct response trends and incorrect response trends 
should be examined before instruction is modified 

7 Correct-to-error ratios are very helpful for gaining a quick 
estimation of a si.udent's learning stage 

8, High trials-at-cnterion standards may be beneficial for the 
student experiencing a persistent learning difficulty 

9 Variability is a dimension of student performance that should be 
examined carefully—the greater the variability, the less in- 
structional control. 
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1 0. When in doubt about whether or not to modify a program either 

(a) collect additional data or 

(b) reexamine the basic data decision rules pertaining 
to trend analysis. 

Summary und-Conclusion 

A growing number of school age children and youth are trapped in a 
failed educational system. Although an empirical approach is not common 
in most classrooms (Tawney, 1984; Wesson et al., 1904), data-based 
instruction is recognized by experts as an important feature of class- 
rooms that successfully serve children with learning and behavioral 
difficulties (Gable etal., 1985). By graphing and visually inspecting pupil 
performance data, teachers can apply decision rules, adjust instruction 
and reduce the time students engage in nonproductive programs. 

This monograph represents an effort to consolidate, for the prac- 
titioner, information on data-based instruction and data-based decision- 
making. As we have discussed, an ample body of research has accumulated 
to guide decisions on when to modify instruction; however, less is known 
about how to modify instruction (Fuchs, 1936). Consequently, the wisdom 
of any instructional decision should be confirmed on the basis of ongoing 
evaluation of pupil performance (Eaton, 1978). By selecting an appropriate 
performance standard and applying the known decision rules, the extent to 
which teachers produce gains in students with histories of academic 
failure is likely to be enhanced . 
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